Recently, the so-called "neural network" model has been used to solve problems in speech recognition, character recognition and expert systems.
Conventionally, one teacher-supervised learning method of a neural network is carried out as follows. One piece of input data is fed into the network and the output value of each output node is calculated. After that, the learning algorithm determines the necessary changes to the weights, and the weights are updated. Some networks accumulate the value for weight changes, and update the weights after all data are fed into the network. Such methods are disclosed, for example, in the following background references: "Statistical Pattern Recognition with Neural Networks: Benchmarking Studies" by Kohonen, G. Barne and R. Chrisley in IEEE, Proc. of ICNN, Vol. I, pp. 61-68, July 1988 and "Learning Internal Representations by Error Propagation," Vol. I of "Parallel Distributed Processing: Explorations in the Microstructure of Cognition" (see especially chapter 8), MIT Press, Cambridge, Mass., 1986, by D. Rumelhart, G. E. Hinton, and R. J. Williams. Each of the foregoing references is incorporated herein by reference.
After the learning procedure, the neural network is able to recognize input data--that is, classify input data which is unknown or unlearned, but which is similar to the learned data, into a proper class, by correlating the unknown input patterns with prelearned patterns. Thus, a neural network constructed according to prior methods can provide a high recognition ability for unlearned data, if a sufficient variety of patterns of learning data are used for learning one class of data which is later to be recognized.
However, when an unlearned character or other datum to be recognized has a feature which is the same as a feature of the learned data, but the feature is located in a different position in the unlearned data, neural networks of prior methods do not provide highly accurate recognition ability. The method called "time-delay neural network" (TDNN) solves this in part, by learning the data in different positions. The method is disclosed in a report "Phoneme Recognition using Time-Delay Neural Networks" by A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. Lang in IEEE Trans. Acoust., Speech, Signal Processing, Vol. 37, pp. 1888-1898, Dec. 1989. (These articles are incorporated herein by reference.) Importantly, though, the Waibel (TDNN) method does not detect the actual locations of the feature in the learned data, and the weights are updated only by all the shifted data. Therefore, learning in the TDNN method is inefficient.
A method for solving a similar problem in set forth in "Handwriting Digit Recognition with a Back-Propagation Network" by Le Cun, et al. in Neural Information Processing Systems, Vol. 2, pages 396-404 (1989), which by Way of background is incorporated herein by reference. The Le Cun architecture uses a five-layer network where the upper layers are used to detect the locations of local features.
Other problems can arise when an input character to be recognized is the same as a learned character, but in a different font, so that the various minute features of the input character are located differently from those of the learned character. In systems wherein entire characters are compared for recognition, the differing locations appear as noise.